Brain interaction during cooperation: Evaluating local properties of multiple-brain network

Subjects’ interaction is the core of most human activities. This is the reason why a lack of coordination is often the cause of missing goals, more than individual failure. While there are different subjective and objective measures to assess the level of mental effort required by subjects while facing a situation that is getting harder, that is, mental workload, to define an objective measure based on how and if team members are interacting is not so straightforward. In this study, behavioral, subjective and synchronized electroencephalographic data were collected from couples involved in a cooperative task to describe the relationship between task difficulty and team coordination, in the sense of interaction aimed at cooperatively performing the assignment. Multiple-brain connectivity analysis provided information about the whole interacting system. The results showed that averaged local properties of a brain network were affected by task difficulty. In particular, strength changed significantly with task difficulty and clustering coefficients strongly correlated with the workload itself. In particular, a higher workload corresponded to lower clustering values over the central and parietal brain areas. Such results has been interpreted as less efficient organization of the network when the subjects’ activities, due to high workload tendencies, were less coordinated

Passive BCI in operational environments: insights, recent advances and future trends

this mini-review aims to highlight recent important aspects to consider and evaluate when passive Brain-Computer Interface (pBCI) systems would be developed and used in operational environments, and remarks future directions of their applications

A new perspective for the training assessment: Machine learning-based neurometric for augmented user's evaluation

Inappropriate training assessment might have either high social costs and economic impacts, especially in high risks categories, such as Pilots, Air Traffic Controllers, or Surgeons. One of the current limitations of the standard training assessment procedures is the lack of information about the amount of cognitive resources requested by the user for the correct execution of the proposed task. In fact, even if the task is accomplished achieving the maximum performance, by the standard training assessment methods, it would not be possible to gather and evaluate information about cognitive resources available for dealing with unexpected events or emergency conditions. Therefore, a metric based on the brain activity (neurometric) able to provide the Instructor such a kind of information should be very important. As a first step in this direction, the Electroencephalogram (EEG) and the performance of 10 participants were collected along a training period of 3 weeks, while learning the execution of a new task. Specific indexes have been estimated from the behavioral and EEG signal to objectively assess the users' training progress. Furthermore, we proposed a neurometric based on a machine learning algorithm to quantify the user's training level within each session by considering the level of task execution, and both the behavioral and cognitive stabilities between consecutive sessions. The results demonstrated that the proposed methodology and neurometric could quantify and track the users' progresses, and provide the Instructor information for a more objective evaluation and better tailoring of training programs. © 2017 Borghini, Aricò, Di Flumeri, Sciaraffa, Colosimo, Herrero, Bezerianos, Thakor and Babiloni

EEG-based mental workload neurometric to evaluate the impact of different traffic and road conditions in real driving settings

Car driving is considered a very complex activity, consisting of different concomitant tasks and subtasks, thus it is crucial to understand the impact of different factors, such as road complexity, traffic, dashboard devices, and external events on the driver’s behavior and performance. For this reason, in particular situations the cognitive demand experienced by the driver could be very high, inducing an excessive experienced mental workload and consequently an increasing of error commission probability. In this regard, it has been demonstrated that human error is the main cause of the 57% of road accidents and a contributing factor in most of them. In this study, 20 young subjects have been involved in a real driving experiment, performed under different traffic conditions (rush hour and not) and along different road types (main and secondary streets). Moreover, during the driving tasks different specific events, in particular a pedestrian crossing the road and a car entering the traffic flow just ahead of the experimental subject, have been acted. A Workload Index based on the Electroencephalographic (EEG), i.e., brain activity, of the drivers has been employed to investigate the impact of the different factors on the driver’s workload. Eye-Tracking (ET) technology and subjective measures have also been employed in order to have a comprehensive overview of the driver’s perceived workload and to investigate the different insights obtainable from the employed methodologies. The employment of such EEG-based Workload index confirmed the significant impact of both traffic and road types on the drivers’ behavior (increasing their workload), with the advantage of being under real settings. Also, it allowed to highlight the increased workload related to external events while driving, in particular with a significant effect during those situations when the traffic was low. Finally, the comparison between methodologies revealed the higher sensitivity of neurophysiological measures with respect to ET and subjective ones. In conclusion, such an EEG-based Workload index would allow to assess objectively the mental workload experienced by the driver, standing out as a powerful tool for research aimed to investigate drivers’ behavior and providing additional and complementary insights with respect to traditional methodologies employed within road safety research

A New Perspective for the Training Assessment: Machine Learning-Based Neurometric for Augmented User's Evaluation

Inappropriate training assessment might have either high social costs and economic impacts, especially in high risks categories, such as Pilots, Air Traffic Controllers, or Surgeons. One of the current limitations of the standard training assessment procedures is the lack of information about the amount of cognitive resources requested by the user for the correct execution of the proposed task. In fact, even if the task is accomplished achieving the maximum performance, by the standard training assessment methods, it would not be possible to gather and evaluate information about cognitive resources available for dealing with unexpected events or emergency conditions. Therefore, a metric based on the brain activity (neurometric) able to provide the Instructor such a kind of information should be very important. As a first step in this direction, the Electroencephalogram (EEG) and the performance of 10 participants were collected along a training period of 3 weeks, while learning the execution of a new task. Specific indexes have been estimated from the behavioral and EEG signal to objectively assess the users' training progress. Furthermore, we proposed a neurometric based on a machine learning algorithm to quantify the user's training level within each session by considering the level of task execution, and both the behavioral and cognitive stabilities between consecutive sessions. The results demonstrated that the proposed methodology and neurometric could quantify and track the users' progresses, and provide the Instructor information for a more objective evaluation and better tailoring of training programs

Multivariate model for cooperation: bridging Social Physiological Compliance and Hyperscanning

The neurophysiological analysis of cooperation has evolved over the past 20 years, moving towards the research of common patterns in neurophysiological signals of people interacting. Social Physiological Compliance (SPC) and Hyperscanning represent two frameworks for the joint analysis of autonomic and brain signals respectively. Each of the two approaches allows to know about a single layer of cooperation according to the nature of these signals: SPC provides information mainly related to emotions, and Hyperscanning that related to cognitive aspects. In this work, after the analysis of the state of the art of SPC and Hyperscanning, we explored the possibility to unify the two approaches creating a complete neurophysiological model for cooperation considering both affective and cognitive mechanisms. We synchronously recorded electrodermal activity, cardiac and brain signals of 14 cooperative dyads. Time series from these signals were extracted and Multivariate Granger Causality was computed. The results showed that only when subjects in a dyad cooperate there is a statistically significant causality between the multivariate variables representing each subject. Moreover, the entity of this statistical relationship correlates with the dyad's performance. Finally, given the novelty of this approach and its exploratory nature, we provided its strengths and limitations

EEG-Based Mental Workload Neurometric to Evaluate the Impact of Different Traffic and Road Conditions in Real Driving Settings

Car driving is considered a very complex activity, consisting of different concomitant tasks and subtasks, thus it is crucial to understand the impact of different factors, such as road complexity, traffic, dashboard devices, and external events on the driver’s behavior and performance. For this reason, in particular situations the cognitive demand experienced by the driver could be very high, inducing an excessive experienced mental workload and consequently an increasing of error commission probability. In this regard, it has been demonstrated that human error is the main cause of the 57% of road accidents and a contributing factor in most of them. In this study, 20 young subjects have been involved in a real driving experiment, performed under different traffic conditions (rush hour and not) and along different road types (main and secondary streets). Moreover, during the driving tasks different specific events, in particular a pedestrian crossing the road and a car entering the traffic flow just ahead of the experimental subject, have been acted. A Workload Index based on the Electroencephalographic (EEG), i.e., brain activity, of the drivers has been employed to investigate the impact of the different factors on the driver’s workload. Eye-Tracking (ET) technology and subjective measures have also been employed in order to have a comprehensive overview of the driver’s perceived workload and to investigate the different insights obtainable from the employed methodologies. The employment of such EEG-based Workload index confirmed the significant impact of both traffic and road types on the drivers’ behavior (increasing their workload), with the advantage of being under real settings. Also, it allowed to highlight the increased workload related to external events while driving, in particular with a significant effect during those situations when the traffic was low. Finally, the comparison between methodologies revealed the higher sensitivity of neurophysiological measures with respect to ET and subjective ones. In conclusion, such an EEG-based Workload index would allow to assess objectively the mental workload experienced by the driver, standing out as a powerful tool for research aimed to investigate drivers’ behavior and providing additional and complementary insights with respect to traditional methodologies employed within road safety research

The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence

Real-world experience is typically multimodal. Evidence indicates that the facilitation in the detection of multisensory stimuli is modulated by the perceptual load, the amount of information involved in the processing of the stimuli. Here, we used a realistic virtual reality environment while concomitantly acquiring Electroencephalography (EEG) and Galvanic Skin Response (GSR) to investigate how multisensory signals impact target detection in two conditions, high and low perceptual load. Different multimodal stimuli (auditory and vibrotactile) were presented, alone or in combination with the visual target. Results showed that only in the high load condition, multisensory stimuli significantly improve performance, compared to visual stimulation alone. Multisensory stimulation also decreases the EEG-based workload. Instead, the perceived workload, according to the "NASA Task Load Index" questionnaire, was reduced only by the trimodal condition (i.e., visual, auditory, tactile). This trimodal stimulation was more effective in enhancing the sense of presence, that is the feeling of being in the virtual environment, compared to the bimodal or unimodal stimulation. Also, we show that in the high load task, the GSR components are higher compared to the low load condition. Finally, the multimodal stimulation (Visual-Audio-Tactile-VAT and Visual-Audio-VA) induced a significant decrease in latency, and a significant increase in the amplitude of the P300 potentials with respect to the unimodal (visual) and visual and tactile bimodal stimulation, suggesting a faster and more effective processing and detection of stimuli if auditory stimulation is included. Overall, these findings provide insights into the relationship between multisensory integration and human behavior and cognition

Neurophysiological characterization of normal hearing and unilateral hearing loss children: a comparison among EEG-based indices for information processing and decision-making levels.

The identification of measurable indices of cerebral functions to be applied in clinical settings is ever more felt as necessary for a more thorough and objective evaluation of patients cognitive performance. In the present paper, the electroencephalographic-based indices of mental workload (WL = frontal θ/parietal α) and of mental engagement (ME = β/(α+θ)), calculated along the brain midline, have been employed to characterize the eventual specific patterns of cerebral activations during a speech in noise perception task in normal hearing (NH) and unilateral hearing loss (UHL) children. Results showed no differences between the groups for the frontal bilateral noise condition (in which both signal and noise were emitted by two loudspeakers placed +45° and -45° in relation to the participant), while in lateralized noise conditions the UHL group showed higher parietal ME values for the Noise to the Deaf Ear condition). Finally, the NH group showed a different distribution of ME values among frontal, central and parietal electrodes, with higher ME values in the central and parietal ones in correspondence of the Noise to the Left Ear condition. The WL index analysis did not provide any significant differences. Results suggest the relevance of including the analysis of the beta rhythm in the neurophysiological assessment of the neural processing of speech in noise stimuli in normal hearing and hearing impaired participants

How neurophysiological measures can be used to enhance the evaluation of remote tower solutions

International audienceNew solutions in operational environments are often, among objective measurements, evaluated by using subjective assessment and judgement from experts. Anyhow, it has been demonstrated that subjective measures suffer from poor resolution due to a high intra and inter operator variability. Also, performance measures, if available, could provide just partial information, since an operator could achieve the same performance but experiencing a different workload. In this study we aimed to demonstrate i) the higher resolution of neurophysiological measures in comparison to subjective ones, and ii) how the simultaneous employment of neurophysiological measures and behavioural ones could allow a holistic assessment of operational tools. In this regard, we tested the effectiveness of an EEG-based neurophysiological index (WEEG index) in comparing two different solutions (i.e. Normal and Augmented) in terms of experienced workload. In this regard, 16 professional Air Traffic Controllers (ATCOs) have been asked to perform two operational scenarios. Galvanic Skin Response (GSR) has also been recorded to evaluate the level of arousal (i.e. operator involvement) during the two scenarios execution. NASA-TLX questionnaire has been used to evaluate the perceived workload, and an expert was asked to assess performance achieved by the ATCOs. Finally, reaction times on specific operational events relevant for the assessment of the two solutions, have also been collected. Results highlighted that the Augmented solution induced a local increase in subjects performance (Reaction times). At the same time, this solution induced an increase in the workload experienced by the participants (WEEG). Anyhow, this increase is still acceptable, since it did not negatively impact the performance and has to be intended only as a consequence of the higher engagement of the ATCOs. This behavioural effect is totally in line with physiological results obtained in terms of arousal (GSR), that increased during the scenario with augmentation. Subjective measures (NASA-TLX) did not highlight any significant variation in perceived workload. These results suggest that neurophysiological measure provide additional information than behavioural and subjective ones, even at a level of few seconds, and its employment during the pre-operational activities (e.g. design process) could allow a more holistic and accurate evaluation of new solutions